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SYSTEM AND METHOD FOR FEATURE LOCATION AND TRACKING IN 
MULTIPLE DIMENSIONS INCLUDING DEPTH 

CROSS-REFERENCE TO RELATED APPLICATIONS 
[0001] This application claims priority under 35 U.S.C. §1 19(e)(1) and 37 C.F.R. 
§ 1.78(a)(4) to U.S. provisional application serial number 60/220,309, entitled 
SYSTEM AND METHOD FOR FEATURE LOCATION AND TRACKING IN 
MULTIPLE DIMENSIONS INCLUDING DEPTH and filed July 24, 2000; and 
claims priority under 35 U.S.C. § 120 and 37 C.F.R. § 1.78(a)(2) as a continuation-in- 
part to U.S. patent application serial number 09/206,195, entitled FACE 
RECOGNITION FROM VIDEO IMAGES and filed December 4, 1998. The entire 
disclosure of U.S. patent application serial number 09/206,195 is incorporated herein 
by reference. 

BACKGROUND OF THE INVENTION 
[0002] The present invention relates to feature tracking techniques, and more 
particularly, to an eye tracking technique that determines the location of a person's 
eyes in three-dimensional space. 

[0003] Virtual reality systems are able to generate three-dimensional images 
viewed by a person without special glasses using, for example, auto-stereoscopic 
imaging. Auto-stereoscopic imaging requires real time determination of a viewer's 
eyes in depth or in three dimensions. 

[0004] Accordingly, there exists a need for a system and related tools for location 
of a person's features in three-dimensional space. The present invention satisfies 
these needs. 
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SUMMARY OF THE INVENTION 
[0005] The present invention is directed to a method and related system for 
determining a feature location in multiple dimensions including depth. The method 
includes providing left and right camera images of the feature and locating the feature 
in the left camera image and in the right camera image using bunch graph matching. 
The feature location is determined in multiple dimensions including depth based on 
the feature locations in the left camera image and the right camera image. 

[0006] Other features and advantages of the present invention should be apparent 
from the following description of the preferred embodiments taken in conjunction 
with the accompanying drawings, which illustrate, by way of example, the principles 
of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0007] FIG. 1 is an elevation view of a face for feature tracking, according to the 
present invention. 

[0008] FIG. 2 is a plan view of the face of FIG. 1 with respect to two tracking 
cameras. 

[0009] FIG. 3 is a schematic diagram of the geometry of the face and tracking 
cameras of FIG. 2. 

[00010] FIG. 4A is an image from a left camera of the face of FIG. 1. 
[00011] FIG. 4B is an image from a right camera of the face of FIG. 1. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[00012] The present invention is directed to a method and related system for 
determining a feature location in multiple dimensions including depth. The method 
includes providing left and right camera images of the feature and locating the feature 
in the left camera image and in the right camera image using bunch graph matching. 
The feature location is determined in multiple dimensions including depth based on 
the feature locations in the left camera image and the right camera image. 

[00013] An embodiment of the invention is described using a simple face image 12 
is shown in FIG. 1. The left eye E of the face image is marked with a small diamond 
to indicate the left eye as a feature for tracking. For simplicity, tracking for only one 
feature is described. However, several features may be tracked by performing the 
analysis below for each feature. 

[00014] The location and tracking of the left eye may be accomplished using two 
cameras, a right camera CR and a left camera CL, as shown in FIG. 2. Two cameras 
are generally required for acquiring the location in multiple dimensions including 
depth using a simple geometrical model shown in FIG. 3. The coordinate system may 
be selected such that the cameras lie along an x-axis and the depth from the cameras 
is measured along a z-axis. The distance to the left eye E along the z-axis is a depth 
D and the distance along the x-axis is a length L (measured from the location of the 
right camera CR). A normal ray from each camera, NR and NL, indicates an image 
ray associated with an approximate center of the cameras imaging area. 

[00015] The imaging areas of the left and right cameras are shown in FIGS. 4 A and 
4B, respectively. Each imaging area is a rectangular array of imaging or picture 
elements (pixels). Each vertical row of pixels in each image area corresponds to a 
particular slope M for an image ray originating at an origin (based on a pinhole 
model) of the respective camera. 


Gray Cary\GT\6248303.1 
103353-1 


Docket No.: EYEM1240-1 


4 


[00016] The cameras may be calibrated for the image ray slope associated with each 
verticle row of pixels. A feature in the image frame may be located and tracked using 
elastic bunch graph matching. As shown in FIG. 4A, the left eye E is imaged in the 
left image along vertical pixel row PL and, as shown in FIG. 4B, the left eye E is 
imaged in the right image along vertical pixel row PR. The pixel rows PL and PR are 
associated with slopes ML and MR, respectively. Accordingly, the location of the left 
eye E is readily calculated in the x-z plane. Elastic bunch graph matching and more 
sophisticated geometrical models and calibration techniques are described in U.S. 
patent application number 09/206,195. 

[00017] In the elastic graph matching technique, an image is transformed into 
Gabor space using a wavelet transformations based on Gabor wavelets. The 
transformed image is represented by complex wavelet component values associated 
with each pixel of the original image. 

[00018] Although the foregoing discloses the preferred embodiments of the present 
invention, it is understood that those skilled in the art may make various changes to 
the preferred embodiments without departing form the scope of the invention. The 
invention is defined only by the following claims. 

WE CLAIM: 
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